Gaussian Process Classiication and Svm: Mean Field Results and Leave-one-out Estimator
نویسندگان
چکیده
In this chapter, we elaborate on the well-known relationship between Gaussian Processes (GP) and Support Vector Machines (SVM). Secondly, we present approximate solutions for two computational problems arising in GP and SVM. The rst one is the calculation of the posterior mean for GP classiiers using a `naive' mean eld approach. The second one is a leave-one-out estimator for the generalization error of SVM based on a linear response method. Simulation results on a benchmark dataset show similar performances for the GP mean eld algorithm and the SVM algorithm. The approximate leave-one-out estimator is found to be in very good agreement with the exact leave-one-out error.
منابع مشابه
Gaussian Processes for Classi cation :
We derive a mean eld algorithm for binary classiication with Gaussian processes which are based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error which is computed with no extra computational cost. We show that from the TAP approach it is possible to derive both a simpl...
متن کاملGaussian Processes for Classification: Mean-Field Algorithms
We derive a mean-field algorithm for binary classification with gaussian processes that is based on the TAP approach originally proposed in statistical physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error, which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a si...
متن کاملLS-SVM Hyperparameter Selection with a Nonparametric Noise Estimator
This paper presents a new method for the selection of the two hyperparameters of Least Squares Support Vector Machine (LS-SVM) approximators with Gaussian Kernels. The two hyperparameters are the width σ of the Gaussian kernels and the regularization parameter λ. For different values of σ, a Nonparametric Noise Estimator (NNE) is introduced to estimate the variance of the noise on the outputs. ...
متن کاملRadius-margin Bound on the Leave-one- out Error of the Llw-m-svm
To set the values of the hyperparameters of a support vector machine (SVM), one can use cross-validation. Its leave-one-out variant produces an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirement. To overcome this difficulty, several upper bounds on the leave-one-out error of the pattern recognition SVM have been derived. The most pop...
متن کاملA Quadratic Loss Multi-Class SVM
Using a support vector machine requires to set two types of hyperparameters: the soft margin parameter C and the parameters of the kernel. To perform this model selection task, the method of choice is cross-validation. Its leave-one-out variant is known to produce an estimator of the generalization error which is almost unbiased. Its major drawback rests in its time requirement. To overcome thi...
متن کامل